Systems and methods for identifying and remediating sound masking

ABSTRACT

Some embodiments of the invention are directed to enabling a user to easily identify the frequency range(s) at which sound masking occurs, and addressing the masking, if desired. In this respect, the extent to which a first stem is masked by one or more second stems in a frequency range may depend not only on the absolute value of the energy of the second stem(s) in the frequency range, but also on the relative energy of the first stem with respect to the second stem(s) in the frequency range. Accordingly, some embodiments are directed to modeling sound masking as a function of the energy of the stem being masked and of the relative energy of the masked stem with respect to the masking stem(s) in the frequency range, such as by modeling sound masking as loudness loss, a value indicative of the reduction in loudness of a stem of interest caused by the presence of one or more other stems in a frequency range.

RELATED APPLICATIONS

This application is continuation of and claims priority under 35 U.S.C.§ 120 to commonly assigned U.S. patent application Ser. No. 16/512,257,filed Jul. 15, 2019, entitled “Systems And Methods For Identifying AndRemediating Sound Masking,” which is a divisional of commonly assignedU.S. patent application Ser. No. 16/250,486, filed Jan. 17, 2019,entitled “Systems And Methods For Identifying And Remediating SoundMasking,” which is a continuation of commonly assigned InternationalPatent Application No. PCT/US2018/034336, filed May 24, 2018, entitled“Systems and Methods For Identifying And Remediating Sound Masking,”which claims priority to commonly assigned U.S. Provisional PatentApplication Ser. No. 62/516,601, filed Jun. 7, 2017, entitled “Systemsand Methods For Identifying And Remediating Sound Masking,”. Theentirety of each of the documents listed above is incorporated herein byreference.

BACKGROUND

Audio production tools exist that enable users to produce high-qualityaudio. For example, some audio production tools enable users to recordone or more audio sources (e.g., vocals and/or speech captured by amicrophone, music played with an instrument, etc.), process the audio(e.g., to master, mix, design, and/or otherwise manipulate the audio),and/or control its playback. Audio production tools may be used toproduce audio comprising music, speech, sound effects, and/or othersounds.

Some computer-implemented audio production tools provide a graphicaluser interface with which users may complete various production tasks onaudio source inputs. For example, some tools may receive audio input andgenerate one or more digital representations of the input, which a usermay manipulate using a graphical user interface, such as to obtaindesired audio output through equalization and/or other operations.

Sound equalization is a technique used by sound engineers to alter asound recording by applying filters to sound in one or more frequencyranges, so as to boost or attenuate spectral portions of a track. Forexample, high-pass filters may be used to boost higher frequencies andattenuates lower frequencies. By contrast, low-pass filters may be usedto boost lower frequencies and attenuate higher frequencies.Equalization may, for example, be used to create the impression thatcertain sounds in a track or multi-track recording are farther or closerthan others.

BRIEF DESCRIPTION OF DRAWINGS

Various aspects and embodiments of the invention are described belowwith reference to the following figures. It should be appreciated thatthe figures are not necessarily drawn to scale. Items appearing inmultiple figures are indicated by the same reference number in all thefigures in which they appear.

FIG. 1 is a plot illustrating a sound masking phenomenon.

FIG. 2 is a flowchart illustrating a representative method foraddressing sound masking, according to some non-limiting embodiments ofthe invention.

FIG. 3A is a block diagram illustrating a representative system foraddressing sound masking, according to some non-limiting embodiments ofthe invention.

FIG. 3B is a block diagram illustrating an example of an excitationpattern generator, according to some non-limiting embodiments of theinvention.

FIG. 4 is a plot illustrating examples of loudness as a function ofexcitation pattern amplitude, according to some non-limiting embodimentsof the invention.

FIGS. 5A-5B are plots illustrating examples of loudness and partialloudness, respectively, according to some non-limiting embodiments ofthe invention.

FIGS. 6A-6D are plots illustrating examples of loudness loss for variousstems, according to some non-limiting embodiments of the invention.

FIG. 7 is a plot illustrating a comparison between effectively mixedstems and poorly mixed stems, according to some non-limiting embodimentsof the invention.

FIGS. 8A-8B schematic diagrams illustrating examples of graphical userinterfaces, according to some non-limiting embodiments of the invention.

FIG. 9 is a block diagram illustrating a computer system, according tosome non-limiting embodiments of the invention.

DETAILED DESCRIPTION

I. Overview

Sound masking occurs when a listener's ability to perceive one sound isnegatively affected by the presence of another sound. For example, soundmasking may occur when a stem of interest is masked by one or more otherstems. (The term “stem” is used herein to mean a digital representationof an audio track in a multi-track recording, although it is to beunderstood that masking may occur in a single-track recording.) This maybe the case when different stems include audio content that compete withone another in a particular frequency range. For example, bass guitarand drums have the tendency to compete with each other in the lowfrequencies. FIG. 1 is a plot illustrating the effects of sound masking.Plot 100 depicts sound pressure level, expressed in decibels (dB), as afunction of frequency. The curve labeled “threshold in quiet” representsa typical frequency-dependent audibility threshold of a human ear in theabsence of masking sounds. Sounds having pressure levels that are abovethe threshold in quiet are audible. For example, tone 102, a sound tonehaving a center frequency of about 150 Hz, has a sound pressure that isabove the threshold at that frequency, and as a result is audible. Bycontrast, sounds having pressure levels that are below the threshold arenot typically audible to humans.

When multiple sounds occur simultaneously, a human's ability to hear aparticular sound may be impaired by the presence of the other sounds,even if the particular sound has a pressure level that is above thethreshold. For example, in FIG. 1, masking tone 104 has a centerfrequency of about 300 Hz. When masking tone 104 occurs, its presenceeffectively increases the threshold of audibility in a frequency rangeproximate to its center frequency. The extent to which the threshold isincreased may depend on the magnitude of the masking tone 104. The curvelabeled “masking threshold” represents the effective threshold ofaudibility when masking tone 104 is played. Tone 102, which is above the“threshold in quiet” and is thus typically audible when no masking tonesare present, is sufficiently close in frequency to the masking tone, andas a result its audibility may be impaired. Even though the presence ofa masking sound does not modify the pressure level of a tone, it maycause a loss of relative loudness with respect to such a tone. That is,the tone is masked by the masking sound.

The Assignee has appreciated that conventional methods for addressingsound masking in audio mixes are often imprecise, error-prone, and time-and labor-intensive, as they rely heavily on a user's ability torecognize when sound masking is occurring, and in which frequencyrange(s). While a sound engineer may have enough experience and talentto recognize that sound masking is occurring, addressing the maskingoften involves significant trial-and-error before he/she can identifythe frequency(ies) at which the masking is occurring, and address themasking so as to produce satisfying sound quality.

Some embodiments of the present invention provide methods and systemsfor enabling a user to easily identify the frequency range(s) at whichsound masking occurs, so that the masking may be addressed, if the userso desires. Some embodiments may therefore provide for an approach toidentifying and addressing sound masking which is more systematic,efficient, and/or accurate than conventional approaches allow.

The Assignee has appreciated that the extent to which a first stem ismasked by one or more second stems in a frequency range depends not onlyon the absolute value of the energy of the second stem(s) in thefrequency range, but also on the relative energy of the first stem withrespect to the second stem(s) in the frequency range. Accordingly, someembodiments of the invention are directed to modeling sound masking as afunction of the energy of the stem being masked and of the relativeenergy of the masked stem with respect to the masking stem(s) in thefrequency range. This may be accomplished, for example, by modelingsound masking as loudness loss, a value indicative of the reduction inloudness of a stem of interest caused by the presence of one or moreother stems in a frequency range.

Some embodiments of the invention are directed to methods foridentifying masking which affects a track in a multi-track recordingcomprising multiple stems, wherein each stem corresponds to a track. Forexample, some embodiments may involve computing the loudness and thepartial loudness of a stem of interest in each of a plurality offrequency ranges. (The term “loudness” is used herein to refer to howstrongly the sound in a stem of interest is perceived by a human whenthe sound occurs in isolation (i.e., in the absence of sound inadditional stems), and the term “partial loudness” is used herein torefer to how strongly the sound in a stem of interest is perceived by ahuman when the sound occurs concurrently with the sound in one or moreadditional stems.) Loudness and partial loudness may be computed in anyof numerous ways. In some embodiments, loudness and partial loudness maybe computed in accordance with a psychoacoustic model, which may definehow humans perceive sound as a function of frequency. One example of apsychoacoustic model which may be used in computing loudness and partialloudness is described in G. Wichern et al., “Quantitative Analysis ofMasking in Multitrack Mixes Using Loudness Loss”, Audio EngineeringSociety (September 2016), which is herein incorporated by reference inits entirety. This paper cites the psychoacoustic model described in P.Kabal's “An Examination and Interpretation of ITU-R BS.1387: PerceptualEvaluation of Audio Quality, 2002 May, 1”, which is available athttp://www.mp3-tech.org/programmer/docs/kabalr2002.pdf. Of course, anysuitable model, whether psychoacoustic in nature or not, may be used tocompute loudness and/or partial loudness.

In some embodiments, loudness and partial loudness for a given stem arecomputed as non-linear functions of one or more excitation patterns,where an excitation pattern is a representation of the distribution ofacoustic excitation at different frequencies. An excitation pattern may,in some embodiments, be computed for a stem after the stem is passedthrough a filter which models the frequency response of the human ear.Loudness may, in some embodiments, be computed as a function offrequency by passing the excitation pattern through afrequency-dependent compressive nonlinearity, wherein the extent towhich compression is applied depends on whether the frequency is belowthe audibility threshold, above an “extremely loud” threshold, orbetween the two extremes. Partial loudness may, in some embodiments, becomputed in a similar fashion as loudness, with the audibility thresholdbeing replaced by a masking threshold, which may be computed from theexcitation pattern for the masking stem(s). The loudness and partialloudness may then, in some embodiments, be integrated over an equivalentrectangular bandwidth (ERB) scale to obtain short-term and long-termloudness values. Loudness loss may then be computed based upon theloudness and partial loudness. For example, in some embodiments,loudness loss may be computed as the difference between loudness andpartial loudness.

Those skilled in the art will appreciate that sound masking is a commonphenomenon which occurs to some extent in almost all multi-trackrecordings, and that it may even be desirable in certain circumstances.As such, some embodiments of the inventions provide techniques foridentifying to a user only the frequency range(s) in which sound maskingis most prevalent, so that the user is not inundated with notificationsof sound masking occurring. For example, in some embodiments, only thefrequency range(s) in which loudness loss exceeds a predeterminedthreshold (e.g., loudness loss in the 99^(th) percentile) may beidentified to the user, so that he/she may apply corrective measures(e.g., via sound equalization) if desired. By identifying the frequencyrange(s) in which loudness loss is most prevalent, some embodiments ofthe invention may enable the user to quickly and easily determine wherecorrective measures will positively affect sound quality.

II. Identifying Problematic Sound Masking

FIG. 2 depicts a representative process 250 for identifying andaddressing sound masking. FIG. 3A depicts a representative system forperforming representative process 250. Representative system 300 may beimplemented using any suitable combination of hardware and/or softwarecomponents. For example, representative system 300 may comprise an audiorecording system, digital audio workstation, personal computer, portabledevice (e.g., tablet, smartphone, gaming console, etc.), and/or anyother suitable system comprising any suitable combination of components.In some embodiments, representative system 300 may include one or moresoftware programs which, when executed, present a graphical userinterface with which a user may interact to invoke certain functions.

Representative process 250 begins in acts 254 and 256, which may beperformed concurrently or at different times. In act 254, a stem ofinterest is received, and in act 256, one or more additional stems arereceived. Each of the stems received in acts 254 and 256 may, forexample, include sound which is generated by a musical instrument, human(e.g., vocals), microphone, computer system, and/or any other suitableimplement(s). For example, each stem may represent a track in amulti-track recording which consists of sound produced by a particularinstrument. For example, the stem received in act 254 may represent arecording of audio generated with a guitar, and two stems received inact 256 may respectively represent recordings of drums and a bassguitar. Of course, the invention is not limited to a mode ofimplementation wherein each stem includes sound from a single instrumentor other implement. For example, a stem may represent a recording ofsounds produced by multiple instruments.

In some embodiments, a time window function may be applied totime-delimit the stems received in acts 254 and 256. In this respect,time window units 302 and 312 ₁ . . . 312 _(N) (FIG. 3A) may apply atime window function to limit the stems to a particular time window. Anysuitable time window function(s) may be used, such as a Hann function,Hamming function, Blackman function, Gaussian function, cosine function,and/or any other suitable function(s). In representative system 300,time window unit 302 time-delimits the stem of interest received in act254, and time window units 312 ₁ . . . 312 _(N) time-delimits the Nadditional stems received in act 256. Time window units 302 and 312 ₁ .. . 312 _(N) need not all apply the same time window function to thestems which are received.

In some embodiments, acts 254 and 256 involve applying a filter whichmodels the frequency response of the human ear to the stems which arereceived. In representative system 300 (FIG. 3A), filters 304 and 314 ₁. . . 314 _(N) are applied. Any suitable filter(s) may be used for thispurpose. For example, some embodiments may use a finite impulse response(FIR) filter which models the frequency response of a human's outer andmiddle ear.

In some embodiments, acts 254 and 256 may involve transforming thereceived stems to the frequency domain. Any suitable transformationtechnique(s) may be used for this purpose. In representative system 300(FIG. 3A), Fourier transform unit 306 and 316 ₁ . . . 316 _(N) computeFourier transforms of the filtered stems. A Fourier transform may becomputed in any suitable way. For example, a fast Fourier transform(FFT) and/or a short-time Fourier transform (STFT) may be used. Itshould be appreciated that in representative system 300, Fouriertransform units 306 and 316 ₁ . . . 316 _(N) need not all employ thesame transformation technique.

In some embodiments, a Fourier transform may have a variable frequencyresolution across the audible frequencies, so that certain frequencysub-bands of interest include a larger number of samples (and thereforegreater frequency resolution) than other frequency sub-bands. Bylimiting the frequency resolution in certain frequency sub-bands, someembodiments of the invention may limit the computational expenseassociated with transformation to the frequency domain. Of course, theinvention is not limited to limiting the frequency resolution in one ormore frequency sub-bands. For example, some embodiments of the inventionmay define the bandwidths of the sub-bands so that each has similarspectral resolution.

In representative system 300, the Fourier transforms of the stemsreceived in the act 256 are combined using mixer 317. Combination of thetransforms may be accomplished in any suitable way. For example, in someembodiments, a summation function may be used.

It should be appreciated that although the depiction of representativesystem 300 in FIG. 3A indicates that the stem received in act 254 andthe stem(s) received in the act 256 are time-delimited, filtered andtransformed in that sequence, the invention is not limited to such animplementation. In some embodiments, the order of these operations maybe changed, and/or one or more operations may be omitted.

Representative process 250 then proceeds to acts 258 and 260, wherein anexcitation pattern is computed for the transform of the stem of interestand for the transforms of the one or more additional stems,respectively. Acts 258 and 260 may be performed concurrently, or atdifferent times. In representative system 300 (FIG. 3A), excitationpattern generator 308 performs act 258, and excitation pattern generator318 performs act 260.

A representative excitation pattern generator (e.g., comprisingexcitation pattern generator 308 and/or 318) is shown in greater detailin FIG. 3B. The excitation pattern generator shown in FIG. 3B comprisesnoise modeling unit 334, frequency spreading unit 336, and timespreading unit 338. The excitation pattern generator shown in FIG. 3B issimilar in some respects to that which is described in the sectionentitled “Pattern Processing” in P. Kabal, “An Examination andInterpretation of ITU-R BS.1387: Perceptual Evaluation of AudioQuality”, Department of Electrical & Computer Engineering, McGillUniversity (2002), available athttp://www.mp3-tech.org/programmer/docs/kabalr2002.pdf, which isincorporated herein by reference in its entirety. In the excitationpattern generator shown in FIG. 3B, noise modeling unit 334 may be usedto model noise of any type. Examples of noise that may be modeled usingmodeling unit 334 include, but are not limited to, white noise, pinknoise, brown noise, blue noise, violet noise and/or grey noise.

Frequency spreading unit 336 may, for example, be used to spread thespectral content of a frequency sub-band through one or more adjacentsub-bands, so that frequency selectivity of auditory filters can beproperly modeled. In some embodiments, frequency spreading unit 336 maycomprise one or more rounded exponential filters. Each roundedexponential filter may, for example, have a center frequency whichranges from 50 Hz to 15 kHz, and the filters may be uniformlydistributed on the equivalent rectangular bandwidth (ERB) frequencyscale. Of course, any suitable spreading function approach(s) mayalternatively, or additionally, be used.

Time spreading unit 338 may be configured to spread the content of thetemporal portions of each received stem to adjacent temporal portions,so as to smooth the stem in the time domain.

It should be appreciated that although the excitation pattern generatorshown in FIG. 3B includes noise modeling unit 334, frequency spreadingunit 336 and time spreading unit 338, the invention is not limited toemploying an excitation pattern generator which performs all of thesefunctions, or the depicted functions in the order described above. Anyone or more of these functions may be omitted in generating anexcitation pattern for the stem of interest and/or for the additionalstem(s), and/or the order of these operations may be different than thatwhich is described above.

At the completion of acts 258 and 260, representative process 250proceeds to act 266, wherein loudness is computed for the excitationpattern computed in act 258, using an audibility threshold and anextreme loudness threshold obtained in act 265. In some embodiments, theaudibility threshold may represent a minimum excitation level that thehuman ear is able to perceive at a particular frequency, and the extremeloudness threshold may, for example, represent the highest level that ahuman ear may tolerate at a particular frequency. The audibility andextreme loudness thresholds may be frequency-dependent parameters, andmay be specified by a user (based on any suitable parameter(s)), ordefined based upon a psychoacoustic model.

In representative system 300 (FIG. 3A), loudness generator 310 is usedto compute loudness for the stem of interest. Loudness may be computedfor the stem of interest in any of numerous ways. In some embodiments ofthe invention, the cross-adaptive Moore-Glasberg loudness model,described in A. J. R. Simpson, “A practical step-by-step guide totime-varying sounds”, Journal of the Audio Engineering Society, 50(5),pp. 331-42 (2002), which is incorporated herein by reference in itsentirety, may be used to compute loudness for the stem of interest. Theloudness which is computed for the stem of interest may representinstantaneous loudness, a time-smoothed long-/short-term loudness,specific loudness, and/or any other suitable loudness measure(s).

Some embodiments of the invention may involve computing loudness for thestem of interest using one or more compressive non-linear functionswhich represent the extent to which sounds at different frequencies areperceived by the human ear. One representative compressive non-linearfunction is illustrated in FIG. 4. Specifically, FIG. 4 depicts a ploton which the loudness of two different frequencies f₁ and f₂ are shownas a function of the amplitude of a corresponding excitation pattern.

In the plot shown in FIG. 4, the loudness corresponding to frequency f₁has three regions, including a first region which includes amplitudesbelow the audibility threshold t_(A1) received in act 265, a secondregion which includes amplitudes between the audibility threshold t_(A1)and the extreme loudness t_(EL1) received in act 265, and a third regionwhich includes amplitudes exceeding the extreme loudness thresholdt_(EL1). Similarly, the loudness corresponding to frequency f₂ also hasthree regions, including a first region which includes amplitudes belowthe audibility threshold t_(A2) received in act 265, a second regionwhich includes amplitudes between the audibility threshold t_(A2) andthe extreme loudness t_(EL2) received in act 265, and a third regionwhich includes amplitudes exceeding the extreme loudness thresholdt_(EL2). At amplitudes below the audibility threshold, loudness exhibitsa slow growth rate with respect to excitation pattern amplitude, so thatincreases in excitation pattern amplitude are barely perceived by ahuman, if at all. At amplitudes between the audibility threshold and theextreme loudness threshold, loudness exhibits a greater growth rate, sothat increases in excitation pattern amplitude are clearly perceived asincreases in loudness by a human. At amplitudes above the extremeloudness threshold, small increases in excitation pattern amplitude giverise to substantial increases in loudness, so that sound in this regionis perceived as being unpleasantly loud by at least some humans. It canbe seen from the plot shown in FIG. 4 that the loudness associated witha particular sound depends not only on excitation pattern amplitude, butalso on the frequency(ies) in which the amplitude is exhibited.

Referring again to representative process 250 (FIG. 2), partial loudnessis computed in act 268, either concurrently or at a different time thanloudness is computed in act 266. In representative process 250, partialloudness is computed based upon the excitation pattern for the stem ofinterest computed in act 258, the excitation pattern for the additionalstem(s) computed in act 260, and a masking threshold obtained in act264. The masking threshold may, for example, represent a threshold ofaudibility as a function of frequency across the audible frequencyrange, so that sound levels below the masking threshold are notperceptible by a human as masking the stem of interest. The maskingthreshold obtained in act 264 may be defined based upon a psychoacousticmodel, and/or in some other fashion (e.g., it may be computed from theadditional stem(s) received in the act 256). Any suitable maskingthreshold(s) may be used. In representative system 300 (FIG. 3A),partial loudness generator 320 computes partial loudness based upon theexcitation pattern for the stem of interest computed in act 258, theexcitation pattern for the additional stem(s) computed in act 260, andthe masking threshold obtained in act 264. Partial loudness may, forexample, be computed using the Moore-Glasberg loudness model referencedabove. As with the loudness computed in act 266, the partial loudnesswhich is computed in act 268 may represent instantaneous loudness, atime-smoothed long-/short-term loudness, specific loudness, and/or anyother suitable partial loudness measure(s).

The loudness computed in act 266 and the partial loudness computed inact 268 may be expressed in any suitable unit(s), such as phons (i.e., adecibel-like unit expressed on a logarithmic scale), sones (i.e.,expressed on a linear scale), etc. In some embodiments, expressingloudness and partial loudness in phons may reduce the computationalexpense associated with determining loudness loss (as described infurther detail below), since phons may be more susceptible than unitslike sones to straightforward mathematical operations (e.g.,subtraction) to compute loudness loss. However, it should be appreciatedthat loudness, partial loudness, and loudness loss may be expressed inany suitable unit(s), as the invention is not limited in this respect.

At the completion of acts 266 and 268, representative process 250proceeds to act 272, wherein loudness loss is computed. Inrepresentative system 300 (FIG. 3A), loudness loss is computed byloudness loss generator 311. Loudness loss may be computed in any ofnumerous ways. In some embodiments, loudness loss may be computed by asthe difference between the loudness computed in act 266 and the partialloudness computed in act 268, so that loudness loss indicates adifference between how strongly the stem of interest is perceived by ahuman in isolation and how strongly the same stem is perceived inconcurrence with the other stem(s). However, loudness loss is notlimited to being computed in this way. For example, in some embodiments,loudness loss may be proportional to, but not equal to, the differencebetween loudness and partial loudness.

It should be appreciated that the invention is not limited to modelingsound masking based solely upon loudness loss. Any suitablecomputational approach(es) may be used to identify sound masking, as theinvention is not limited in this respect.

Since loudness and partial loudness may, in some embodiments of theinvention, vary as a function of frequency, in some embodiments loudnessloss may vary as a function of frequency as well. That is, the stem ofinterest received in act 254 may be masked in certain frequency range(s)by the other stem(s) received in act 256, and not masked in otherfrequency ranges. Additionally, it should be appreciated that loudnessloss may vary over time. For example, the extent to which the stem ofinterest received in act 254 is masked in certain frequency range(s) bythe other stem(s) received in act 256 may vary over time by transientand/or intermittent sounds in the other stem(s).

It should further be appreciated that each track of a multi-trackrecording may be masked to a certain extent by one or more other tracksin the recording. For example, in a recording which includes tracks thatrespectively include audio produced by a bass, a guitar, a drum set, andvocals, the bass track may be masked by the drum track, the guitar trackmay be masked by the vocal track, the drum track may masked by theguitar and vocal tracks, etc. Accordingly, some embodiments of theinvention provide for computing loudness loss (e.g., using the portionof representative process 250 described above) for each track of amulti-track recording. For example, the loudness and partial loudnessmay be computed for the stem corresponding to each track, and loudnessloss may be computed as the difference between the loudness and partialloudness of the stem.

At the completion of act 272, representative process 250 (FIG. 2)proceeds to act 276, wherein the frequency(ies) at which maskingsatisfies a particular threshold are identified to a user, based uponthe loudness loss computed in act 272 and a loudness loss thresholdobtained in act 274.

Any suitable loudness loss threshold may be used. In this respect, asnoted above, the Assignee has appreciated that sound masking is a commonphenomenon that occurs to some extent in almost all multi-trackrecordings. As such, in some embodiments, a loudness loss threshold maybe selected which results in the user only being notified of a subset ofthe sound masking events which actually occur, to avoid inundating theuser with notifications that such masking events have occurred. Forexample, in some embodiments, a loudness loss threshold is used whichresults in only the masking events with associated loudness loss in the99^(th) or greater percentile being identified to the user. Of course,any suitable threshold(s) may be used, and each threshold may be definedin any suitable way. For example, a loudness loss threshold need not bebased upon the magnitude of the loudness loss associated with a maskingevent. As one example, some embodiments may use a loudness lossthreshold which is based at least in part upon the amount of time duringwhich masking occurs, so as to avoid identifying fleeting and/orone-time sound masking events to the user. As another example, someembodiments may use a threshold which is based at least in part upon aloudness loss of particular magnitude occurring over a particular amountof time. Any suitable threshold(s) may be used.

Additionally, recognizing that some users may be more sensitive to soundmasking than others, some embodiments of the invention may allow a userto manually define a loudness loss threshold. As another example, someembodiments may provide for a loudness loss threshold to be defined foreach of a population of users based upon preferences demonstrated by thepopulation. For example, machine learning techniques may be used todetermine how user-defined loudness loss thresholds change over time,and set a threshold accordingly. Any of numerous techniques may be used.

The range of frequencies in which sound masking satisfying a thresholdis identified in act 276 may be any suitable range. For example, in someembodiments, sound masking is identified in act 276 across the spectrumof frequencies which are audible to humans. Of course, the invention isnot limited to being implemented in this manner. Sound masking may beidentified in any one or more suitable frequency spectra.

Any suitable technique(s) may be used to identify loudness loss whichsatisfies a loudness loss threshold to a user. For example, a graphicaluser interface may be used, such as in the manner described below withreference to FIGS. 8A-8B.

At the completion of act 276, representative process 250 proceeds to act278, which involves enabling a user to apply one or more correctivemeasures to address loudness loss. Examples of corrective measuresinclude filtering, mixing, equalization, attenuation, amplification,panning, compression, and other operations. Such measures may be appliedmanually by a user (e.g., via a graphical user interface, one example ofwhich is described below), may be applied automatically, and/or usingmay be applied using some combination of manual and automatictechniques. For example, in some embodiments, in response to identifyinga particular quantity of loudness loss events occurring within aparticular frequency range over a particular period of time, one or morelevels associated with a masked stem and/or one or more masking stemsmay automatically be applied.

At the completion of act 278, representative process 250 completes.

It should be appreciated from the foregoing that some embodiments of theinvention, by applying quantitative analysis techniques to theidentification of sound masking, may enable a user to quickly and easilyidentify the frequency range(s) in which masking is occurring so thatthe masking may be addressed. By contrast, in conventional approaches,the user is forced to rely upon his/her experience and skill, and apainstaking trial-and-error process, to identify sound masking. As such,some embodiments of the invention may reduce the amount of time that auser may spend processing a body of audio content, make the audioproduction process more productive and pleasurable for the user overall,and free up the computational resources that the user may otherwise haveexpended trying to identify where sound masking is occurring for othertasks.

III. Loudness Loss as a Measure of Sound Masking

Some illustrative examples of loudness and partial loudness (e.g.,computed according to representative process 250) are shown in FIGS. 5Aand 5B, respectively. In the plots depicted in these figures, loudnessand partial loudness (each expressed in phons) are shown as a functionof frequency. FIG. 5A includes four loudness curves, including curve 501for a stem representing audio produced by a bass guitar, curve 502 for astem representing audio produced by vocals, curve 503 for a stemrepresenting audio produced by a guitar, and curve 504 for a stemrepresenting audio produced by a drum set. FIG. 5B includes four partialloudness curves 511, 512, 513 and 514, which correspond to the stemsrepresented by curves 501, 502, 503 and 504, respectively.

It can be seen in FIG. 5A that the loudness of the bass guitar is muchgreater than the loudness of the other instruments at frequencies belowapproximately 200 Hz. It can also be seen that the difference betweenthe loudness represented by curves 502, 503 and 504 in FIG. 5A and thepartial loudness represented by corresponding curves 512, 513 and 514 inFIG. 5B is much greater than the difference between the loudnessrepresented by curve 501 and the partial loudness represented bycorresponding curve 511, indicating that the loudness loss for theseinstruments (i.e., computed as the difference between loudness andpartial loudness) is significant in that frequency range. That is, thelarger difference between the loudness represented by curves 502, 503and 504 and the partial loudness represented by corresponding curves512, 513 and 514 than the difference between the loudness represented bycurve 501 and the partial loudness represented by curve 511 indicatesthat the bass guitar masks the vocals, guitar and drums in thisfrequency range.

It can also be seen that that the loudness of the vocals and the guitarrepresented by curves 502 and 503 is greater than the loudness of thebass guitar represented by curve 501 at frequencies between 500 Hz and 4kHz. It can also be seen that the difference between the loudness of thebass guitar represented by curve 501 and the partial loudness of thebass guitar represented by corresponding curve 511 is not so great inthis frequency range. However, it can further be seen from the “dip” incurve 512 at approximately 3.5-4.0 kHz that the difference between theloudness of the vocals represented by curve 502 and the partial loudnessof the vocals represented by curve 512 becomes significant in thisfrequency range, indicating that the vocals are being masked by theguitar.

These phenomena are further illustrated in FIGS. 6A, 6B, 6C and 6D,which are plots showing loudness loss for stems representing the bassguitar, vocals, guitar and drums, respectively, for over a hundredmulti-track recordings. Specifically, curve 601 in FIG. 6A representsthe median loudness loss for the bass guitar, curve 602 in FIG. 6Brepresents the median loudness loss for the vocals, curve 603 in FIG. 6Crepresents the median loudness loss for the guitar, and curve 604 inFIG. 6D represents the median loudness loss for the drums. The darkshaded regions surrounding the curve representing median loudness lossin each plot represent loudness loss between the 25th percentile and the75^(th) percentile, and the light shaded regions bordering loudness lossbetween the 25th percentile and the 75^(th) percentile representloudness loss between the 1^(st) and 5^(th) percentile, and between the95^(th) and 100^(th) percentile. It can be seen from curves 602-604that, as described above with reference to FIGS. 5A-5B, the loudnessloss for the vocals, guitar and drums is significant in frequenciesbelow about 200 Hz, while the loudness loss for the bass guitar is lowin this frequency range, thus indicating that the bass guitar commonlymasks the vocals, guitar and drums in this frequency range.

It can also be seen from curve 602 that there is an increase in medianloudness loss for the vocals at approximately 3.5-4.0 kHz, and fromcurve 603 that there is a corresponding decrease in median loudness lossfor the guitar at approximately 3.5-4.0 kHz, indicating that vocals arecommonly masked by the guitar in this frequency range.

One aspect of the invention lies in the Assignee's recognition that (1)a certain amount of sound masking may be inevitable whenever there aremultiple instruments represented in a multi-track recording, asillustrated by the median loudness loss curves in FIGS. 6A-6D showing atleast some loudness loss throughout the frequency spectrum for all ofthe instruments represented; and (2) at least some of this sound maskingmay not be correctable via equalization. To identify the sound maskingwhich a user e.g., a professional sound engineer) may considerproblematic, and to determine the extent to which such masking iscorrectable, the Assignee compared a set of “effectively” mixed stems toa set of “poorly” mixed stems. Specifically, the “effectively” mixedstems were taken from the MedleyDB, which is a dataset of annotated,royalty-free multitrack recordings for non-commercial use and academicresearch. As the recordings in the MedleyDB were mixed by professionalsound engineers, it was assumed that the corresponding stems were mixed(i.e., had equalization parameters set) effectively. “Poorly” mixedstems were created by randomly modifying various equalization parametersfor the same tracks.

FIG. 7 is a plot which shows loudness loss as a function of frequencyfor the “effectively” and “poorly” mixed stems. In particular, FIG. 7shows median loudness loss and loudness loss in the 99^(th) percentilefor both types of stems. It can be seen in FIG. 7 that the difference inloudness loss between “effectively” and “poorly” mixed stems isrelatively small at the median, indicating that any difference inloudness loss between “effectively” and “poorly” mixed tracks may not becorrectable through straightforward adjustment of equalizationparameters. It can also be seen in FIG. 7, however, that the differencein loudness loss at the 99^(th) percentile between “effectively” and“poorly” mixed tracks is more significant, suggesting that this loudnessloss may be correctable via equalization. The fact that loudness loss inthe 99^(th) percentile or greater is of much greater magnitude thanloudness loss at the median suggests also that it may be the loudnessoccurs which users may consider problematic.

As a result of this recognition, some embodiments of the inventionemploy a loudness loss threshold which is defined by the magnitude ofloudness loss for a masking event being in the 99^(th) percentile orgreater, to identify which masking events are to be identified to a user(e.g., defined in act 274 of representative process 250 shown in FIG.2). It should be appreciated, however, that any suitable loudness lossthreshold may be used. It should also be appreciated that a loudnessloss threshold need not be based on the magnitude of loudness loss. If athreshold based upon the magnitude of loudness loss is used, thethreshold may be defined in any suitable way. For example, such athreshold may be set so as to identify to the user only the maskingevents having associated loudness loss in the 95^(th) percentile ofgreater, in the 90^(th) percentile or greater, in the 80^(th) percentileor greater, and/or to any other suitable value(s).

IV. Enabling a User to Address Sound Masking

As noted above, some embodiments of the invention are directed toenabling a user to view and/or remediate sound masking, as the userdesires. For example, some embodiments may provide a graphical userinterface with which a user may view information on masking affectingone or more tracks in a multi-track recording, and apply one or morecorrective measures to address sound masking in one or more stems.

FIGS. 8A-8B depict a representative graphical user interface 800 fordisplaying information on sound masking affecting a track in amulti-track recording, as the multi-track recording is being replayed inreal time, so that the user may apply corrective measures as desired.Graphical user interface 800 includes area 806 for displaying a curverepresenting a stem of interest and area 810 for displaying a curverepresenting one or more masking stems. In the example depicted, thecurves in areas 806 and 810 show sound pressure as a function offrequency.

Graphical user interface 800 includes area 802 for displayinginformation on sound masking. Specifically, area 802 displays a bargraph indicating a number of loudness loss events satisfying a loudnessloss threshold that have occurred as the multi-track recording wasreplayed in each of a plurality of frequency ranges. In the exampleshown, the height of each bar in the bar graph corresponds to the numberof loudness loss events satisfying a loudness loss threshold (e.g., asdetermined by performing representative process 250 shown in FIG. 2)which occurred at the considered frequency. It can be seen in FIG. 8Athat the bars labeled 802 ₂ indicate a number of loudness loss eventsroughly akin to a bell curve occurring within a particular frequencyband; that the bars labeled 802 ₁ indicate a number of loudness lossevents occurring within another frequency band, with the greatest numberof loudness loss events occurring at the lowest frequency within theband; and that the bar labeled 802 ₃ indicates a significant number ofloudness loss events occurring within a localized frequency range.Although graphical user interface 800 uses a bar graph to identifyloudness loss events, any suitable representation(s) may be used toconvey information on sound masking to a user.

In some embodiments, the bar graph shown in area 802 may indicate anumber of loudness loss events occurring within a particular period oftime prior to display. For example, the bar graph may indicate a numberof loudness loss events satisfying a loudness loss threshold whichoccurred 0.5 second prior to display, 1 second prior to display, 3seconds prior to display, etc. Use of a rolling time interval in thismanner may allow a user to see how the way in which sound maskingaffects a stem at different frequencies varies over time. It may alsoensure that the information provided to the user is timely (e.g., itdoes not relate to sound masking which occurred ten seconds ago), sothat any corrective measures the user may desire are applied at thepoint in time when they are most beneficial.

Additionally or alternatively, some embodiments of the invention mayonly identify loudness loss events to a user if the associated maskingoccurs for at least a threshold amount of time. In this way, theseembodiments may avoid needlessly notifying the user of fleeting orone-time masking events. As an example, in some embodiments the bargraph shown in area 802 may only reflect a loudness loss event in aparticular frequency range if the event occurred for at least 0.5seconds, for at least 1 second, for at least 3 seconds, etc. Anysuitable amount of time may be used for this purpose.

Using the representative graphical user interface shown in FIG. 8A, auser may quickly and easily identify the frequency ranges in which soundmasking is most pronounced, rather than relying on an inefficienttrial-and-error process to do so. As a result, the user may quicklyfocus any corrective action desired on the affected frequency ranges,saving time and computational resources, and making the audio productionprocess more enjoyable and satisfying overall.

Graphical user interface 800 includes controls for applying suchcorrective measures. Specifically, graphical user interface 800 includescontrols for adjusting equalization parameters for the stem of interestand for the masking stem(s). These controls are provided in controlareas 804 and 808, respectively. In some embodiments, these controls mayenable a user to apply filters to boost or attenuate the signal for thestem of interest and/or the masking stem(s) in certain frequency ranges.In the example shown, the controls for the stem of interest includethose labeled 804 ₁, 804 ₂, 804 ₃, 804 ₄, 804 ₅, and 804 ₆, the controlsfor the masking stem(s) include those labeled 808 ₁, 808 ₂, 808 ₃, 808₄, 808 ₅, and 808 ₆. It should be appreciated that any other suitablenumber of filter controls may be used to adjust equalization parametersfor the stem of interest and for the masking stem(s).

In some embodiments, a user may adjust equalization parameters for thestem of interest by dragging (e.g., using a mouse, touch screen, voicecontrols, etc.) one or more of controls 804 _(n) to change the positionof the control(s) within control area 804, and for the masking stem(s)by dragging a control 808 _(n) to change the position of the control(s)within control area 808. For example, the user may (for example) use acursor provided by graphical user interface 800 to adjust the positionof a control within a control area. An example is shown in FIG. 8B.Specifically, FIG. 8B shows that controls 804 ₁, 804 ₂, 804 ₃, 804 ₄,and 804 ₅ have been moved from their original locations within controlarea 804 shown in FIG. 8A. In the example shown, this movement isperformed to boost the stem of interest signal in frequency ranges whereloudness loss has occurred, by moving a control 804 _(n) upward (therebyincreasing the amplitude of a corresponding filter) and/or laterally toa location proximate the considered frequency range (thereby shiftingthe center frequency of the corresponding filter to within the range)within control area 804. FIG. 8B also shows that controls 808 ₁, 808 ₂,808 ₃, and 804 ₄ have been moved from their original locations withincontrol area 808 shown in FIG. 8A. Specifically, controls 808 _(n) havebeen moved downward (thereby decreasing the amplitude of a correspondingfilter) and/or laterally to locations proximate the frequency rangeswhere loudness loss has occurred (shifting the center frequency of eachcorresponding filter to within the range), so as to attenuate themasking stem(s) signal in the considered frequency ranges. Of course, auser may modify the location of a control 804 _(n) or 808 _(n) toachieve objectives other than remediating loudness loss within afrequency range. For example, the user may modify the location of acontrol to apply filters to improve perceived sound quality.

Although not shown in FIGS. 8A-8B, in some embodiments, a graphical userinterface may provide controls which enable a user to modify a loudnessloss threshold, so that more loudness loss events (e.g., if the loudnessloss threshold is adjusted downward) or less loudness loss events (e.g.,if the loudness loss threshold is adjusted upward) are identified to theuser. A graphical user interface implemented in accordance with theinvention may provide any suitable controls for performing any suitableaudio production-related tasks.

V. Implementation Detail

It should be appreciated from the foregoing that some embodiments of theinvention may be implemented using a computer. For example,representative system 300 (FIGS. 3A-3B) may be implemented at least inpart using a computer. FIG. 9 depicts a general purpose computingdevice, in the form of a computer 910, which may be used to implementcertain aspects of the invention. In computer 910, components include,but are not limited to, a processing unit 920, a system memory 930, anda system bus 921 that couples various system components including thesystem memory to the processing unit 920. The system bus 921 may be anyof several types of bus structures including a memory bus or memorycontroller, a peripheral bus, and a local bus using any of a variety ofbus architectures. By way of example, and not limitation, sucharchitectures include Industry Standard Architecture (ISA) bus, MicroChannel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnect (PCI) bus also known as Mezzanine bus.

Computer 910 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 910 and includes both volatile and nonvolatile media,removable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media includes both volatileand nonvolatile, removable and non-removable media implemented in anymethod or technology for storage of information such as computerreadable instructions, data structures, program modules or other data.Computer storage media include, but are not limited to, RAM, ROM,EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical disk storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other one or more media which may be used to store the desiredinformation and may be accessed by computer 910. Communication mediatypically embody computer readable instructions, data structures,program modules or other data in a modulated data signal such as acarrier wave or other transport mechanism and includes any informationdelivery media. The term “modulated data signal” means a signal that hasone or more of its characteristics set or changed in such a manner as toencode information in the signal. By way of example, and not limitation,communication media include wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media. Combinations of the any of the aboveshould also be included within the scope of computer readable media.

The system memory 930 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 931and random access memory (RAM) 932. A basic input/output system 933(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 910, such as during start-up, istypically stored in ROM 931. RAM 932 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 920. By way of example, and notlimitation, FIG. 12 illustrates operating system 934, applicationprograms 935, other program modules 939, and program data 937.

The computer 910 may also include other removable/non-removable,volatile/nonvolatile computer storage media. By way of example only,FIG. 12 illustrates a hard disk drive 941 that reads from or writes tonon-removable, nonvolatile magnetic media, a magnetic disk drive 951that reads from or writes to a removable, nonvolatile magnetic disk 952,and an optical disk drive 955 that reads from or writes to a removable,nonvolatile optical disk 959 such as a CD ROM or other optical media.Other removable/non-removable, volatile/nonvolatile computer storagemedia that can be used in the exemplary computing system include, butare not limited to, magnetic tape cassettes, flash memory cards, digitalversatile disks, digital video tape, solid state RAM, solid state ROM,and the like. The hard disk drive 941 is typically connected to thesystem bus 921 through an non-removable memory interface such asinterface 940, and magnetic disk drive 951 and optical disk drive 955are typically connected to the system bus 921 by a removable memoryinterface, such as interface 950.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 12, provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 910. In FIG. 12, for example, hard disk drive 941 isillustrated as storing operating system 944, application programs 945,other program modules 949, and program data 947. Note that thesecomponents can either be the same as or different from operating system934, application programs 935, other program modules 539, and programdata 937. Operating system 944, application programs 945, other programmodules 949, and program data 947 are given different numbers here toillustrate that, at a minimum, they are different copies. A user mayenter commands and information into the computer 910 through inputdevices such as a keyboard 992 and pointing device 991, commonlyreferred to as a mouse, trackball or touch pad. Other input devices (notshown) may include a microphone, joystick, game pad, satellite dish,scanner, or the like. These and other input devices are often connectedto the processing unit 920 through a user input interface 590 that iscoupled to the system bus, but may be connected by other interface andbus structures, such as a parallel port, game port or a universal serialbus (USB). A monitor 991 or other type of display device is alsoconnected to the system bus 921 via an interface, such as a videointerface 990. In addition to the monitor, computers may also includeother peripheral output devices such as speakers 997 and printer 999,which may be connected through a output peripheral interface 995.

The computer 910 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer980. The remote computer 980 may be a personal computer, a server, arouter, a network PC, a peer device or other common network node, andtypically includes many or all of the elements described above relativeto the computer 910, although only a memory storage device 981 has beenillustrated in FIG. 12. The logical connections depicted in FIG. 12include a local area network (LAN) 971 and a wide area network (WAN)973, but may also include other networks. Such networking environmentsare commonplace in offices, enterprise-wide computer networks, intranetsand the Internet.

When used in a LAN networking environment, the computer 910 is connectedto the LAN 971 through a network interface or adapter 970. When used ina WAN networking environment, the computer 910 typically includes amodem 972 or other means for establishing communications over the WAN973, such as the Internet. The modem 972, which may be internal orexternal, may be connected to the system bus 921 via the user inputinterface 990, or other appropriate mechanism. In a networkedenvironment, program modules depicted relative to the computer 910, orportions thereof, may be stored in the remote memory storage device. Byway of example, and not limitation, FIG. 12 illustrates remoteapplication programs 985 as residing on memory device 981. It will beappreciated that the network connections shown are exemplary and othermeans of establishing a communications link between the computers may beused.

Embodiments of the invention may be embodied as a computer readablestorage medium (or multiple computer readable media) (e.g., a computermemory, one or more floppy discs, compact discs (CD), optical discs,digital video disks (DVD), magnetic tapes, flash memories, circuitconfigurations in Field Programmable Gate Arrays or other semiconductordevices, or other tangible computer storage medium) encoded with one ormore programs that, when executed on one or more computers or otherprocessors, perform methods that implement the various embodiments ofthe invention discussed above. As is apparent from the foregoingexamples, a computer readable storage medium may retain information fora sufficient time to provide computer-executable instructions in anon-transitory form. Such a computer readable storage medium or mediacan be transportable, such that the program or programs stored thereoncan be loaded onto one or more different computers or other processorsto implement various aspects of the present invention as discussedabove. As used herein, the term “computer-readable storage medium”encompasses only a tangible machine, mechanism or device from which acomputer may read information. Alternatively or additionally, theinvention may be embodied as a computer readable medium other than acomputer-readable storage medium. Examples of computer readable mediawhich are not computer readable storage media include transitory media,like propagating signals.

Having thus described several aspects of at least one embodiment of thisinvention, it is to be appreciated that various alterations,modifications, and improvements will readily occur to those skilled inthe art. Such alterations, modifications, and improvements are intendedto be part of this disclosure, and are intended to be within the spiritand scope of the invention. Further, though advantages of the presentinvention are indicated, it should be appreciated that not everyembodiment of the invention will include every described advantage. Someembodiments may not implement any features described as advantageousherein and in some instances. Accordingly, the foregoing description anddrawings are by way of example only.

Various aspects of the present invention may be used alone, incombination, or in a variety of arrangements not specifically discussedin the embodiments described in the foregoing, and it is, therefore, notlimited in its application to the details and arrangement of componentsset forth in the foregoing description or illustrated in the drawings.For example, aspects described in one embodiment may be combined in anymanner with aspects described in other embodiments.

The invention may be embodied as a method, of which various exampleshave been described. The acts performed as part of the method may beordered in any suitable way. Accordingly, embodiments may be constructedin which acts are performed in an order different than illustrated,which may include different (e.g., more or less) acts than those whichare described, and/or which may involve performing some actssimultaneously, even though the acts are shown as being performedsequentially in the embodiments specifically described above.

Use of ordinal terms such as “first,” “second,” “third,” etc., to modifyan element does not by itself connote any priority, precedence, or orderof one claim element over another or the temporal order in which acts ofa method are performed, but are used merely as labels to distinguish oneelement having a certain name from another element having the same name(but for use of the ordinal term) to distinguish the elements.

Also, the phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including,” “comprising,” or “having,” “containing,” “involving,” andvariations thereof herein, is meant to encompass the items listedthereafter and equivalents thereof as well as additional items.

The invention claimed is:
 1. At least one computer-readable mediumhaving instructions recorded thereon which, when executed in a computingsystem, cause the computing system to perform a method comprising actsof: (A) receiving a multi-track audio recording comprising a pluralityof tracks synchronized over a time interval, a first track of theplurality of tracks corresponding to a first audio recording of themulti-track audio recording, a second track of the plurality of trackscorresponding to a second audio recording of the multi-track audiorecording, the first track and the second track comprising soundproduced by different instruments; (B) determining a loudness of thefirst audio recording at each of a plurality of frequencies during aparticular time period of the time interval; (C) determining a loudnessof the second audio recording at each of the plurality of frequenciesduring the particular time period; (D) determining a partial loudness ofthe first audio recording at a first frequency of the plurality offrequencies during the particular time period, based at least in part onthe loudness of the second audio recording, at the first frequencyduring the particular time period, determined in the act (C); (E)determining a loudness loss for the first audio recording at the firstfrequency during the particular time period based at least in part onthe partial loudness determined in the act (D); and (F) creating aprocessed multi-track audio recording by applying one or more measuresto the multi-track audio recording received in the act (A), based uponthe loudness loss determined in the act (E).
 2. An apparatus,comprising: at least one computer processor, programmed to: receive amulti-track audio recording comprising a plurality of trackssynchronized over a time interval, a first track of the plurality oftracks corresponding to a first audio recording of the multi-track audiorecording, a second track of the plurality of tracks corresponding to asecond audio recording of the multi-track audio recording, the firsttrack and the second track comprising sound produced by differentinstruments; determine a loudness of the first audio recording at eachof a plurality of frequencies during a particular time period of thetime interval; determine a loudness of the second audio recording ateach of the plurality of frequencies during the particular time period;determine a partial loudness of the first audio recording at a firstfrequency of the plurality of frequencies during the particular timeperiod, based at least in part on the determined loudness of the secondaudio recording, at the first frequency during the particular timeperiod; determine a loudness loss for the first audio recording at thefirst frequency during the particular time period based at least in parton the determined partial loudness; and create a processed multi-trackaudio recording by applying one or more measures to the receivedmulti-track audio recording, based upon the determined loudness loss. 3.The apparatus of claim 2, wherein the at least one computer processor isprogrammed to transform a digital representation of the first audiorecording to the frequency domain, and to generate a first excitationpattern using a result of the transforming.
 4. The apparatus of claim 3,wherein the at least one computer processor is programmed to transformthe digital representation of the first audio recording at least in partvia a Fourier transform having a variable frequency resolution acrossthe plurality of frequencies.
 5. The apparatus of claim 2, wherein theat least one computer processor is programmed to determine the loudnessof the first audio recording at each of the plurality of frequenciesusing an audibility threshold and an extreme loudness threshold, theaudibility threshold representing a minimum excitation level that ahuman ear perceives at each of the plurality of frequencies, the extremeloudness threshold representing a highest excitation level that a humanear tolerates at each of the plurality of frequencies.
 6. The apparatusof claim 2, wherein the at least one computer processor is programmed todetermine loudness comprising at least one of an instantaneous loudness,a time-smoothed long-/short-term loudness, and a specific loudness. 7.The apparatus of claim 2, wherein the at least one computer processor isprogrammed to determine a partial loudness of the first audio recordingat a first frequency of the plurality of frequencies during the timeperiod based at least in part on (1) the determined loudness of thesecond audio recording during the given time period at the firstfrequency and (2) a masking threshold representing a minimum thresholdof human audibility at each of the plurality of frequencies.
 8. Theapparatus of claim 7, wherein the masking threshold is based at least inpart upon a psychoacoustic model.
 9. The apparatus of claim 2, whereinthe at least one computer processor is programmed to determine partialloudness comprising at least one of an instantaneous loudness, atime-smoothed long-/short-term loudness, and a specific loudness. 10.The apparatus of claim 2, wherein the at least one computer processor isprogrammed to determine loudness at the first frequency measured inphons, determine partial loudness at the first frequency measured inphons, and determine a loudness loss by subtracting the determinedpartial loudness from the determined loudness.
 11. The apparatus ofclaim 2, wherein the at least one computer processor is programmed tocause display of a graphical user interface depicting the determinedloudness loss, the graphical user interface enabling the user to provideinput for at least partially remediating the loudness loss.
 12. Theapparatus of claim 11, wherein the input relates to one or more measurescomprising filtering, mixing, equalizing, attenuating, amplifying,panning, compressing and phase rotating at least one of the firstdigital representation and the second digital representation at thefirst frequency.
 13. The apparatus of claim 2, wherein the instrumentproducing the sound included in each of the first track and the secondtrack is a musical instrument, human vocals, a microphone, or a computersystem.
 14. The apparatus of claim 2, wherein the at least one computerprocessor is programmed to apply one or more measures specified via userinput.
 15. The apparatus of claim 2, wherein the at least one computerprocessor is programmed to automatically apply one or more measures. 16.The apparatus of claim 2, wherein at least one computer processor isprogrammed to: receive a multi-track audio recording comprising morethan two tracks synchronized over a time interval, each trackcorresponding to a different audio recording, each track comprisingsound produced by a different instrument; determine a loudness of eachof the audio recordings other than the first audio recording at each ofthe plurality of frequencies during the particular time period; anddetermine a partial loudness of the first audio recording at a firstfrequency of the plurality of frequencies during the particular timeperiod, based at least in part on the determined loudness of another ofthe audio recordings, at the first frequency during the particular timeperiod.